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ABSTRACT 


This study examines the ARB decision and reporting process 


of the seven "A" schools and the Advanced Electronics School 


from the Service School Command in San Diego, Ca. Three 


research questions are addressed: 


1. 


Do differences exist between the schools! 7B prece- 
dures which could promote either inaccurate or decreased 
standardization in attrition reporting? 


How much agreement is there among the ARB members 
concerning the evaluation of student factors for a given 
type decision? 


What other information can be captured that sheds light 
on the attrition process? 


The results from the study revealed several items: 


1. 


2 


ARB procedures differ across schools. 


There is moderate to high levels of agreement concerning 
the value certain student factors bring to the decision 
process. 


Some of the ARB members perceive that the command policy 


has changed from producing high quality technicians to 
reducing their school's attrition rates. 
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I. INTRODUCTION 


The Department of Defense (DoD) provides a wide variety of 
training that ranges from recruit training for initial 
acclimation to military life, to professional development and 
education. Active and reserve student training loads in DoD 
have steadily increased from 225,900 in 1980 to an anticipated 
total of 233,900 for FY91. [Ref. 1l:p. 7] The Navy's student 
loads have remained fairly steady since 1985 with a current 
projected load of 66,200 for FY 91 and FY92. [Ref. lip. 6] 

Navy enlisted training comprises two major areas, recruit 
training and specialized skill training. Specialized skill 
training is further divided into initial skill training and 
progression skill training. [Ref. lip. V-4] Initial skill 
training is the equivalent to Navy "A" school training. For 
the most part, new recruits do not already posses the skills 
required by the Navy to operate effectively. Therefore, most 
graduates from Recruit Training Commands (RTCs) go directly to 
"A" schools to receive specialized skill training. 

"A" school training is the most cost-effective means of 
training recruits for most of their initial assignments in the 
fleet. [Ref. 2:p. 141 The projected student input totals for 
"A" school attendance are 128,045 for FY91 and 126,603 for 
FY92. However, the projected graduation totals are 117,411 


and 116,161, respectively, for the same time period. fRef. 





lip. V-5]} The difference of the two totals (over 10,000 
students for each year) represents a Significant loss of 
resources in the form of student drops from training. 

"A" school attrition affects the Navy in several ways. 
First, managers of enlisted ratings need a reliable source of 
newly-trained sailors to replace the fleet sailors who are due 
for rotation or discharge. Additionally, "A" school attrition 
requires recruiting commands to achieve higher recruit totals 
in ord to maintain acceptable rate end-strength levels. 
Those losses in training and increases in recruiting goals 
represent a significant loss in resources. Lost resources 
include student pay, student travel costs, instructor and 
school administrator pay, as well as the lost productivity of 
the students and instructors while the students are in the 
training status. 

"2" school attrition has its most deleterious affect on 
enlisted ratings that require pipeline training courses to 
achieve a final rating classification. An individual 
designated for a particular rating ma’’ attend several schools 
before reaching his/her final duty station. The sum of the 
attrition at those schools equals the pipeline attrition rate. 
Considering the average attrition rate for an initial skill 
school is eight percent, the sum of four schools with an 
average attrition rate would result in a pipeline attrition 


rate of 32 percent. 








In relation to "A" schools, "C" school attrition is not a 
major concern. The current average of "C" school attrition is 
three percent. There are some significant differences between 
"A" school students and "C" school students, which might 
explain the differences in average attrition rates. "AN 
school students usually come directly from RTCs, most are 18 
to 20 years of age, they have not fuily acclimated themselves 
to the military way of life they may not possess the 
discipline to succeed in tae military training environment, 
and as the training progress: s they may question their desire 
toe obtain that particula: edad. Conversely, "C" school 
students usually core fro. fleet units, they have already 
completed their initial skill training, and have added fleet 
experience tc that knowledge hase. These students have been 
closely evaluated by their suneriors in the fleet and have 
keen recov-nended for follow-on training. Consequently, when 
"co" schocl students arrive for training, most have the 
maturity, discipline, and motivation to complete their course 
ef instruction. 

Thus, the Navy's concern for uottrition from specialized 
skill training is focused primarily on the "A" schoois where 
student losses are the highest. Considerable attention has 
been paid to reducing "A" school attrition through research 
and development, and a working group was formed to address the 


proklen. 











Most of the research on "A" school attrition has looked at 
the effects that student characteristics such as mental 
ability anu level of education have on academic and 
nonacademic attrition. (Refs. 3,4] Other studies have 
examined the role technical and non-technical courses have on 
academic and nonacademic attrition patterns, as well as the 
proportion of students dropped from training who ere then 
reclassified into another rating or are sent directly to the 
fl-et as an un-designated general detail (GENDET). (Refs. 
5,67 

In 1987, the Chief of Naval Education and Training (CNET), 
established a "Model Schcols Program." [Ref. 7] The intent 
cf the program was to improve training by 

...-bringing available resources into contact with a Navy 
school and collectively working together with the school 
manage ent staff to identify problems that impede school 
success and develop solutions that can be implemented by the 
school staff. {Ref. 8} 
The EM-A school was designated as the first model scnool with 
the idea that activities used to improve training in this 
environment could be transitioned to othe: technical training 
schools. Given the broad spectrum of improvements introduced 
to training in the model schools context, reduced attrition 
would not be an unexpected result for the schools that become 
part of the prog.am. 
Another CNET group, the Training Efficacy Quality 


Management Board, sponsors and distributes a lessons-]earned 


letter to its training commands. The letter is a compilation 





of the effective actions taken by the commands that have been 
successful in curbing their schools' attrition. The letter 
allows the schools to review the actions taken by other 
schools. (Ref. 9] 

The purpose of the present research effort is to explore 
how decisions are made to drop students from training, and to 
determine if there is standardization in the decision making 
process. Accuracy and standardization in these decisions are 
essential in order to develop appropriate programs to reduce 


attrition. 


fy. BACKGROUND 

Navy training commands provide instruction for a wide 
variety of technical and experience levels. Training levels 
range from apprentice training "A" schools to NEC-producing 
"Cc" schools and highly technical "F" schools. The Chief of 
Naval Education and Training (CNET), and the Chief of Naval 
Technical Training (CNTT) have provided instructions to their 
subordinate Commands concerning all training-related issues. 
However, individual Training Commands may adjust that guidance 
to fit their particular environment. Each individual "A," 
"C" or "F" school has a Directcr of Training, who reports to 
either the Officer in Charge of Training or the Commanding 
Officer of the training command, depending on the size of the 
command. The Director of Training is responsible for the 


administraticn and performance of the school. 


"A" school student enrollment is comprised primarily of 
students coming directly from the Recruit Training Commands 
(RTCs), although a small percentage comes from fleet units. 
Each student must meet some minimum entrance requirement to 
be admitted into the school--usually a minimum total Armed 
Forces Vocational Aptitude Battery (ASVAB) score or a minimum 
score on one of the ASVAB sub-tests. The newly reporting 
student participates in school indoctrination, which may last 
from several days to three weeks depending on the school and 
the student. Part of the indoctrination phase may require a 
battery of reading and arithmetic tests. The schools use 
these tests to determine the student's knowledge of the basic 
skills, which are necessary to complete the training program. 
Students who do not pass these exams remain in the 
indoctrination rhase and receive remediation on their skill 
deficiencies. Students who fail remediation do not continue 
with the training pipeline and eventually are reassigned to 
other duty. Students who continue with training are assigned 
to an academic class and a military company. 

The military side of "A" school training is the "A" School 
Military Training Company (ASMT), formally known as the 
Integrated Training Battalion (ITB). The ASMTs provide the 
students with a continuation of the general military training 
and physical fitness conditioning they received as recruits. 
Students assigned to the ASMT must muster with their divisions 


at predesignated times; students living with their spouses do 








not report for the first muster. The student's responsibili- 


ties to the ASMT company include space cleanliness, muster- 
ing, room and personnel inspections, marching to meals and 
class as a division, and physical training. Also, the 
students perform watch-standing and collateral duties that 
will prepare them for their follow-on fleet assignments. 
{Ref. 10:pp. 3-1--5-3] 

ASMTs may be composed of several companies with each tied 
directly to a specific academic school. The "A" School 
company commander is usually a Chief Petty Officer. The ASMT 
company commanders report to their respective Directors of 
Training or Division Officers. The "A" School companies 
maintain a reccrd on each student consisting of all the 
military training the student receives, any violations of 
school policy, military deficiencies, and any other personal 
information that may seem necessary. If a student commits an 
offense against school policy, he/she may go before a military 
review board (MRB). The MRBs are used to correct the 
student's military deficiencies; however, if the problem 
persists, the board can recommend the student be dropped from 
training. Offenses of a more serious nature are handled at 
Officer in Charge Mast or Captain's Mast; both can result in 
dropping a student from training. 

The academic curriculum of an "A" school may be divided 
into phases. The school may have both military instructors, 


usually Second Class Petty Officers and above, and contracted 


~) 


civilians. The instructors teach just one segment of the 
training phase. A training phase may be made up of many 
training segments. Students are tested upon completion of 


each segment of training. Students who fail the test receive 
remediation including a reexamination. Students who pass 
remediation usually continue with their class, students who 
fail receive a mandatory Academic Review Board (ARB). [Ref. 
ll:pp. 2-3] 
1. Academic Review Boards 

"All Apprentice Training, Class ‘'AP,' 'A,' and NEC 
awarding 'C' schools are required to convene ARBs’ for 
attrition and setback decisions." [Ref. ll:p. 2] The stated 
primary function of the ARB is to enhance student academic 
progress and to deter student failure. Along with that 
primary duty, the board must make an unbiased determination of 
whether the student has the motivation and ability to complete 
the training. {Ref. ll:p. 1} An ARB is initiated by the 
student's Phase Chief at the request of the student's 
instructor for a given training segment. An ARB may be 
convened for any one of the following five reasons: (1) the 
student is recommended for acceleration through training; (2) 
the student fails to meet course learning objectives and after 
remedial study fails the retests; (3) the student continually 
fails course learning objectives, even though he/she passes 
all remedial exams; (4) the student's laboratory performance 


is consistently below standard; or (5) the student fails 





remediation for a learning objective following an ARB 


recommendation to continue with the class. [Ref. ll:pp. 2-3] 
The ARB consists of four members chosen from the 
instructional staff, which includes officer and enlisted 
instructional/supervisory personnel, classroom and learning 
center instructors, and education/training specialists. 
Supervisory personnel who have command designated authority 
for approval/disapproval of ARB recommendations may not sit 
as members of the ARB. Those personnel responsible for 
completing student personnel and reclassification actions may 
not sit as ARB members. {Ref. ll:pp. 1-2} The board is 
chaired by the senicr member and there are two other voting 
members as well as one non-voting recorder. After examining 
all the available relevant data, the board conducts an 
interview with the student. During the interview, the board 
attempts to gain information on any problems the student may 
be having that could cause the student's academic problems. 
Once the ARB is satisfied that they have reviewed all the 
available information, the board makes a recommendation on the 
student's future training status. If the board is able to 
determine the student's problem, it can initiate appropriate 
corrective actions. However, the board must make one of four 
recommendations: (1) continue the student with class, with or 
without remediation; (2) setback to the next class, with or 
without remediation; (3) accelerate to the next class; or (4) 


drop from training. 


wo 





If the board's recommendation is to drop the student 
from training, it must also make a further recommendation to 
transfer the student directly to the fleet as a general detail 
(GENDET), reclassify the student to another rating, or 
separate the student from service. For the board to recommend 
that the student continue with the class or setback to another 
Class, there must be clear evidence that the student has the 
ability and the motivation to complete the training. To 
recommend a student be dropped from training, the student must 
show an unwillingness or an inability to complete the 
training. [Ref. ll:p. 5] 

The ARB's recommendation is accompanied by a student 
action code (SAC). The SAC is a three digit code that 
indicates the type of action taken by the board and in those 
cases when a student is dropped from training, the SAC will 
also indicate why the student was dropped. The student action 
codes were expanded on 1 October 1990 by CNET to give a more 
accurate picture of school attrition patterns. The SACS are 
separated into two categories, academic and nonacademic. The 
nonacademic category is further separated into sub-categories 
for motivation, medical, and administrative and disciplinary 
reasons. When a school drops a student from training, the 
board may give an academic SAC or a nonacademic SAC from the 
motivation category. 

All academic student actions require an ARB, and CNTT 


Instruction 1540.46A provides guidance about special 
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circumstances that may cause training commands confusion about 
whether to take the student to an ARB or not. 
Any decision made concerning a student's training status 
which is made without the student being present should be 
construed as an administrative action and not an action of 
an ARB. Such administrative actions should be limited to 
cases involving legal, medical, and administrative holds or 
interruption of training, and to cases dealing with 
unauthorized absences, other disciplinary offenses or 
attrition which required by higher authority, such as 
fraudulent enlistment or other administrative or medical 
causes. [Ref. ll:p. 2] 
After the ARB makes a recommendation, the student's case is 
forwarded up the chain of command. Setbacks can be approved 
by the Director of Training. Recommendations to drop a 
student must be approved by the Commanding Officer. If the 
ARB recommendation is approved by the Commanding Officer, the 
SAC is entered into the Navy Integrated Training Administra- 
tive System (NITRAS) database, which is updated daily. 

NITRAS is the database for all Navy training. It 
consists of four sub-files; the master course reference file, 
the student master file, the training summary file, and the 
Pipeline management file. The database provides Naval 
training administrators a means to track the progress of 
students' training through their Naval careers. The data can 
be aggregated by school to determine attrition and setback 
patterns for every course offered. 

The accuracy of the attrition information that is 


input into the database is directly related to the care of the 


ARB procedures and the proper assignment of SACs by the 


il 





individual training commands. The concern over the accuracy 
of the NITRAS database is reflected by the following extract 
from a CNTT instruction: 

The accuracy and care involved in student coding at the ARB 

level is fundamental to the creation of an accurate and 

meaningful data base of attrition information. This 

information may be used as a basis for administrative and 

management decisions and research studies, and it may 

influence assignment procedures. [Ref. ll:p. 3] 
The accuracy of these data will be addressed in this thesis 
in the context of decision making at ARBs. 

2. Decision Making 
Ideally, to achieve accurate and standard decision 
making from one ARB to the next, there would be a black box 
into which all the student information could be put, and it 
would generate an appropriate decision that could be repeated 
‘from student to student. However, Naval training schools do 
not have a black box, they rely on a small group to decide 
what appropriate action should be taken concerning a given 
student. Because of this, individual differences and group 
dynamics become a part of the decision making process. 
The problems associated with group decision making are 

well documented. In one example concerning the "Bay of Pigs" 
incident in 1961, several influential members of President 


Kennedy's advisory group had negative information or ill 


feelings about the undertaking but because of the group 


dynamics (e.g., a subtle pressure toward conformity), never 
voiced their concerns. (Ref. 12:p. 136] The dynamics that 
12 


may affect group decision making can be separated into three 


factors: (1) input variables that have potentially negative 
effects on group processes, (2) conformity, and (3) 
polarization. Input variables include task norms, group 


communication patterns, perceived status of other members, 
individual personality characteristics, and group experience. 
{Ref. 13] For example, the group decision making process may 
be affected through inadequate knowledge of the organization's 
goals, thereby focusing a group's actions on the wrong input 
variables. Poor communication patterns within a group may 
induce some members to unknowingly withhold information. An 
individual's self-esteem and the degree to which he/she values 
others' opinions can lead to other types of problems 
asscciated with group decision making like conformity and 
polarization. Conformity affects individuals by shifting 
their opinions to be more like those of the group. 
Polarization works by driving the group member's opinion in 
the direction of his or her initial opinion. {[Ref. 13:pp. 
136-137] 

The problems confronting group decision making would 
sur2ly apply to the ARB process. During the ARB, each voting 
member has one vote and, thus, technically equal power to 
influence the board's recommendation. Ideally the board's 
decision is made free of any influences other than those 
brought to the board, i.e., the student's academic record and 


the results of the board's interview with the student. 
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However, there may be other influences, some outside the board 
and some within the board, that affect the board's ultimate 
recommendation. For example, serving as the chairman may give 
that individual some influence over the other board members. 
By instruction, the chairman is the senior member present and 
may be an immediate superior to one of the other board 
members. One board member may influence other board members 
because of his strength of conviction. One or more of the 
board members may empathize with the student's plight due to 
having Similar demographic characteristics or background. 
Those people to whom we believe ourselves similar elicit an 
empathetic response, whereas those to whom we do not believe 
ourselves similar do not. {Ref. 13:pp. 89-92] Individuals 
with approval/disapproval authority may influence the board's 
decision through actions on past board recommendations, or 
through other forms of communication of their wishes. 
Perceptions of the school's effectiveness by external and 


internal organizations may influence the board's decision. 


B. RESEARCH QUESTIONS 

The present research was initiated because of CNTT's 
concern over the accuracy and standardization of attrition 
reporting by their "A" schools and NEC~awarding "C" schools. 
Accuracy in attrition reporting is essential to maintaining 


an effective NITRAS database. 
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ARBs have a finite group of decisions to make on a 
Student's training status, the most significant being to 
setback the student or drop the student from training for 
either an academic or a nonacademic reason. The distinction 
between academic and nonacademic reasons is’ significant 
because it may affect whether a student will be permitted to 
reclassify into another rating or transfer to the fleet as a 
GENDET. Furthermore, any variation in the decision making 
process among different boards, schools, and training commands 
will affect the accuracy of the NITRAS database. This could 
create unintended results when changes are implemented in a 
school to address a perceived academic or nonacademic 
attrition problem. 

In response to CNTT's concerns, three questions are 
examined: 

1. Do differences exist among the schools' ARB procedures 
that could promote either inaccuracy or decreased 
standardization in attrition reporting? 

2. How much agreement is there among the ARB members 
concerning the evaluation of student characteristics for 
a given decision? 


3. What other information can be captured that sheds light 
on the attrition ARB decision? 


By determining how school personnel make their decisions 
during the ARB process, information can be acquired that will 
help improve attrition reporting accuracy and standardize 


attrition reporting among the various CNTT controlled schools. 








II. METHODOLOGY 


A. SUBJECTS 

The subjects of this study were the ARB members from eight 
"A" Schools and the Advanced Electronics School (AES), which 
incorporates many of the electronic "C" Schools. Seven of the 
"A" Schools and AES are part of Service School Command 
(SERVSCHLCOM), in San Diego, California. The other "A" School 
is Data Systems "A" (DS-A) school, which is part of Combat 
Systems Technical School Command located at Mare Island, 
California. SERVSCLCOM, is the largest training command that 
provides initial skill training on the West Coast. It is home 
for the following "A" Schools: Radioman "A" (RM~A), Interior 
Communications "A" (IC-A), Data Processor "A" (DP-A), Mess 
Specialist "A" (MS-A), Molder "A" (ML-A), Pattern Maker "A" 
(PM-A) and Machinery Repairman "A" (MR=-A). The output from 
the seven "A" Schools represents 20 percent of the FY89 
graduation total for all CNTT controlled "A" Schools. (Ref. 
14) RM-A and IC-A are among 15 "A" schools that have the 
highest attrition rates. 

The study surveyed 91 ARB members, who represented varying 
percentages of their school's instructional staff. The 
representation ranged from 5.3 percent from the AES to 71.0 
percent for IC-A school. Appendix A provides a complete list 


of the school’s instructional staff representation. However, 
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not all the instructional staff may be qualified to sit on an 


ARB. Discussion with the various school representatives 
indicated that the study captured most of the eligible ARB 
members associated with the respective schools. 

The ARB members averaged 14 years of service with 25 
months at their present command. Forty seven percent of the 
respondents were Second Class and First Class Petty Officers, 
32 percent were Chief Petty Officers, 17 percent were Senior 
Chief and Master Chief Petty Officers, and four percent were 
Warrant or Line Officers. Additionally, their ARB experience 
averaged over 75 ARBs and those who had experience as the 
cnairman had ovzr 40 ARBs in that capacity. 

The ARB members were separated into four groups for the 
analysis (see Table 1) hecause several schools (ML-A, MR-A, 
PLi-A, DP-#) are smail and have few ARB members. The criteria 
for grouping the schools were based on those schools that have 
a Similar chain-of-command, i.e., those schools that reperted 
to the same Officer in Charge (SIC). At SERVSCHLCOM, ML-A, 
MS-A, MR-A, AND PM-A are classified as 330% level schools and 
report to the same OIC; IC-A and CP-A also have a similar 
chain-of-command and report to tne same OIC. The information 


gathered from DS-A school was evaluated only with respect to 


ARB procedures. 





TABLE 1 


SCHOOL GROUPINGS 


Group 1 RM-A 

Group 2 IC-A, DP-A 

Group 3 (3300 Level) MR-A, MS-A, PM-A, ML-A 
Group 4 AES 


B. QVUESTIONNATIRE 

The ARB members responded to a 29 item questionnaire, 
which is presented in Appendix B. The design of the 
questicnnaire was based on information gathered from two sets 
of interviews with the ARB members from DS-A School. The 
first set of interviews was conducted to gather information 
“about ARB procedures. The second set of interviews included 
observation of s¢cveral ARBs and follow-up questions between 
each board to further define the student characteristics being 
considered when evaluating each student. Uncertainties in the 
wording of some questions on the questionnaire were examined 


with assistance from CNTT. 


Cc. PROCEDURE 

The questionnaires were administered to ARE members at the 
four groups of schools on separate days. The respondents 
received an initial briefing on the purpose of the study, with 
specific emphasis on the study cbjectives. Subjects were 


encouraged to add relevant information that hau not been 
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included on the questionnaire. No time limit was set on 
completing the questionnaire, however, all respondents had to 
complete it before leaving the room. 

The first four questions required the respondents to 
evaluate nine student characteristics shown in Table 2, 


specifically focusing on the value each factor contributes to 


the decision making process. The values for the responses 
ranged from 1 = not at all =important--rarely used, to 
5 = extremely important--critical factor. The nine factors 
were evaluated over four possible types of decisions: (1) to 


drop a student from training for academic reasons, (2) to drop 
a student from training for non-academic reasons, (3) to 
setback a student for academic reasons, and (4) to setback a 
student for non-academic reasons. 

The remaining 25 questions asked the respondents to 
elaborate on how they judge certain student characteristics, 
how their decision process works, and how their organization's 


ARB and attrition reporting system works. 
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TABLE 2 


STUDENT CHARACTERISTICS CONSIDERED BY ARB MEMBERS 


1. Academic record 

2. Military record 

3. Personal information about the student 
4. ASVAB scores 

5. Amount of night study 

6. Recommendations made to the board 


7. Professional judgement about whether 
the student will make a good sailor 


8. Student attitude/motivation 


$. High school graduate (or not) 


acadrec 
milrec 
persinfo 
asvab 
nghtstdy 


rectobrd 


prfjdgment 
stdtmot 


hsdg 





III. RESULTS AND DISCUSSION 


The results of this study are presented in the context of 
the three study questions proposed in the introduction: 

1. Do differences exist among the schools' ARB procedures 
that could promote either inaccuracy or decreased 
standardization in attrition reporting? 

2. How much agreement is there among the ARB members 
concerning the evaluation of student factors for a given 
ARB decision? 

3. What other information can be captured that sheds light 
on the attrition process? 

A. PROCEDURAL DIFFERENCES 

A number of distinct differences were found among the 
schools' ARB procedures, which could potentially affect the 
standardization of attrition reporting. The information 
addressed in this section was gathered in part from general 
discussions with school personnel. Additionally, some of the 
data generated by the questionnaire that was administered (see 
Appendix B) are presented here. Differences among specific 
schools are included where they are relevant to the 
discussion. 

Who is present at a student's ARB may affect the board's 
decision. In addition to the required board composition, some 


schools have the student's instructor present to elaborate on 


his/her recommendations concerning the student's academic 








abilities and to answer other specific questions that may shed 
more light on the student's problems. 

Just as important in influencing a board's decisions are 
those incidents that limit the board's make-up, i.e., assign- 
ing one to two individuals as permanent chairman for all ARBs. 
The chairman of an ARB has positional power over the other 
board members and in that capacity might exert more influence 
over the board's decision. Therefore, the ARB's decisions 
over time could reflect individual biases that those members 
would consistently bring to each ARB. 

The manner of conducting an ARB may also affect a board's 
decision. Some of the schools conduct their ARBs in a 
relaxed, congenial atmosphere. This is done in an attempt to 
promote a more open discussion with the student, which would 
allow the board to accurately assess the student's problems 
and future potential. Some schools use a checklist to 
evaluate their ARB's conduct and procedures. This evaluation 
by the school's instructional staff occurs several times each 
month. 

The length of an ARB can vary from board to board, the 
shortest board may last only a couple of minutes, while some 
boards last as long as one and a half hours. Table 3 shows 
the average length of time an ARB usually lasts for a given 
student. On average, 70 percent of the boards lasts between 


10 and 30 minutes, 26 percent lasts longer than 30 minutes, 
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while approximately three percent of the boards lasts less 


than ten minutes. 


TABLE 3 


AVERAGE LENGTH OF AN ARB 
(Question 13) 


During an ARB, about how much time is spent on each student? 
Give a range (max/min) and an average. 


less than 10 minutes 3.5% 
10 to 20 minutes 36.8% 
20 to 30 minutes so436 
30 to 40 minutes 13.8% 
greater than 40 minutes 12.6% 


* Percentages based on 8&7 responses. 


As noted earlier, the student's time is divided between 
acadenic classes and duties in the ASMTs. However, not all 
schools consider the student's military performance to the 
same extent at an ARB. Some schools have ASMT instructional 
staff sit on the ARBs as voting members. These schools 
believe they receive more information and can make a more 
accurate determination about the student's training status. 

Another indication of a school's evaluation of the 
student's military performance is the amount of communication 
between the academic sections and the ASMTs. From Table 4, it 
can be seen that RM-A school and IC/DP-A school have the least 


amount of communication with their associated ASMTs. 
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Conversely, the 3300-level schools have considerable 
communication with their ASMT. The AES reported the most 
communication with their associated military side, however, 
the "C" schools do not have ASMTs. They may be responding 
with respect to their accessibility to the student's service 


record. 


TABLE 4 


COMMUNICATION BETWEEN ACADEMIC SCHOOLS 
AND ASMTs (Question 25) 


How much communication is there here between the Military 
Training Divisions and the academic sections on student 
progress? 


RM-A IC/DP-A 3300 AES 
None 61.5% 14.6% 6.2% 11.1% 
- Occasional 38.5% 56.2% 25.0% 0% 
Considerable 0% 29.2% 68.8% 88.9% 
N 13 41 16 iS) 


One of the important decisions a board must face is 
whether to setback a student or drop that student from 
training. Usually, before a student is dropped from training 
he/she will be setback at least once. The number of setbacks 
a student receives will vary depending on the individual case. 
However, the likelihood of setting back a student may also 


vary by the school. Table 5 presents the number of setbacks 
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a board member would give to a student before deciding to drop 


that student from training. 


TABLE 5 


MAXIMUM NUMBER OF SETBACKS 
A BOARD MEMBER WOULD GIVE (Question 27) 


What is the maximum number of setbacks you would give any 
student? 


RM-A IC/DP-A 3300 AES 
No setbacks 8.3% 2.1% 11.8% 9.1% 
One setback 8.3% 0 35.3% 45.4% 
Two setbacks 58.3% 38.3% 29.4% 36.4% 
Three setbacks 25.0% 57.5% 23.5% 9.1% 
More than three ce) 2.1% 0 0 
N 12 47 17 11 


The table shows that the majority of the RM-A school members 
prefer to give the student two setbacks. The IC/DP-A school 
favors setting the student back three times before dropping 
that student from training. The 3300-level schools and the 
AES are less certain, with both schools slightly favoring just 
one setback before dropping the student. In some cases, when 
it seems obvious that the board will decide to setback the 
student, the student is physically setback to the next class 
prior to the board convening. 

The last item a board must consider is the assignment of 


the SAC. The accuracy of assigning the SAC is essential for 
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maintaining an accurate NITRAS database. However, Table 6 
shows a wide range of knowledge about student action codes 
across the schools. The table has three categories: (1) No 
problems with SACs, (2) found SACS confusing, and (3) no 
knowledge of SACs. Many of the ARB members who found the SACs 
hard to use had problems with only a small number of the many 


SACs provided. 


TABLE 6 


ARB MEMBER'S KNOWLEDGE 
OF STUDENT ACTION CODES (Question 19) 


Which of the SACs are confusing or hard to use in any way, 
and why? 


RM-A IC/DP-A 3300 AES 
No problems 
with SACs 25.0% 52.6% 53.3% 20.0% 
SACs are 
confusing 8.3% 23.7% 26.7% 0% 
Do not use 
or have no 
knowledge 66.7% 23.7% 20.0% 80.0% 
N 12 38 15 10 


Some of the lack of knowledge concerning the SACs may be 
due to the fact that at some ARBs the chairman is the only 
member who assigns the SAC based on board recommendations. 
Additionally, some of the board members stated that they did 
not evaluate their students for nonacademic reasons and 


therefore could not give a nonacademic SAC even though the 
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nonacademic SAC may have been more appropriate for that 


particular student. Although, at one of those schools which 
the board members reported not using nonacademic SACs, the 


chairman does assign nonacademic SACs. 


B. AGREEMENT IN DECISION MAKING 

Besides procedural differences, differences in the ARB 
members' opinions about the importance of certain student 
factors for making a decision about a student may also 
contribute to either a lack of standardization or inaccurate 
attrition reporting. These factors were mentioned in Chapter 
II, e.g., the student's academic record, recommendations made 
to the board, etc. If there is little agreement among the 
board members, then it is highly likely that there is a great 
amount of variation in attrition reporting. There is a finite 
amount of information the ARB has available by which it can 
evaluate a student. Therefore, differences between the ARB's 
weighing of a student factor from one set of ARB members to 
another may elicit varied decisions. 

Two sets of analyses were performed on the data. The 
first analysis consisted of the Friedman Two-Way Analysis of 
Variance by Ranks. This test determined whether the ARB 
members value all the student factors equally or if they give 
some factors more weight than others. The values for the nine 
student factors were ranked for each ARB member and then the 


mean rank for each student factor was calculated. In this 


case the most highly valued student factor cculd receive a 
9.0, while the least valued student factor could receive a 
1.0. From the mean ranks, a test statistic with an 
approximate Chi-square distribution was calculated with the 


following formula: 


(5) - 3N(k + 1) 


where: 
N = number of ARB members, 


k 


number of student factors, and 
R = sum of ranks in jth column. [Ref. 15:pp. 167-171} 
The second analysis run was the Kendall's W Coefficient of 
Concordance. Kendall's W served as a measurement of agreement 
among the raters concerning the importance of the given 
student factors. A Kendall coefficient of W = 1.0 woulda 
indicate that there is perfect agreement among the raters. 
Conversely, a coefficient of W = 0 would indicate that there 
is no agreement. The procedures for calculating Kendall's W 
are similar to the Friedman test except that after the mean 
ranks are calculated, the sum of squares of the observed 
deviations from the mean ranks are calculated. From that 
information, Kendall's W may be calculated by the following 


formula: 








Ss = sum of squares of the observed deviations from 
the mean of R, 


of: aes 


k = number of ARB members. 


N 


number of student factors judged. 
Ri = (Sum of ranks; j°= 2 to-9, ‘“[Ref. 152pp.-229=237] 
The analysis of agreement data will be presented for each of 
the four types of decisions from command-wide rankings, and 
then repeated by individual school groups. 
1. Analysis of Command-Wide Responses 
The first set of analyses considered the responses 
command-wide, i.e., schools were combined. The results are 
shown in Table 7. The Friedman's Test indicated that the ARB 
members do not value all of the student factors equally. In 
other words, there are significant differences in the level of 
importance assigned to student factors for each of the four 
types of decisions. The data in Table 7 show the relative 
rank followed by the mean rank of each student factor for a 
given decision (1-9 with 1 as the most important for the 
relative rank). In order to make easier comparisons of the 
mean ranks with the relative rankings (i.e., a relative 
ranking of 1.0 = the most important student factor), the mean 


rankings were subtracted from 10.0. The discussion of the 
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data in Table 7 will begin with the rankings of the student 
factors, followed by a more detailed discussion of specific 
student factors, and finally a discussion of the levels of 


agreement among the ARB members. 


TABLE 7 


RELATIVE AND MEAN RANKINGS 
OF STUDENT FACTORS FOR TYPE OF DECISION 


Academic Nonacademic Academic Nonacademic 

Student setback setback drop drop 
acadrec 2/2.77 6/5.17 1/2.24 6/5.42 
milrec 7/5.68 2/3.59 7/5.80 2/2044 
persinfo 4/4.89 3/3.87 5/5.04 3/3.73 
asvab 8/6.23 8/7.40 8/6.44 8/7.41 
nghtstdy 3/4.70 7/6.05 3/4.54 7/6.38 
rectobrd 5/5.10 4/4.09 4/4.88 5/4.61 
prfjdgmnt 6/4.26 5/4.51 6/5.36 4/4.49 
stdtmot 1/2.29 1/2.39 2/2.54 1/2.27 
hsdg 9/7.97 9/7.93 9/8.16 9/8.26 
N &9 76 88 75 
df 8 & & 8 
Friedman's 

Chi square 270.04 269.84 209.06 345.28 
Kendall's 

Coefficient -42 .50 -50 -64 
Chi square 302.222 301.44 354.88 385.36 


‘All values are significant p < .01. 
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The ARB mevrers ranked the student factors similarly 
for similar type decisions, i.e., the student factors for 
academic setback and academic drop from training had similar 
ranks, as did the ranks for the nonacademic setback and 
nonacademic drop from training type decisions. For the 
acaaemic type decisions, the student's academic record and 
attitude/motivation received similar mean ranks and were 
ranked much h.gher than any other student factor considered. 
In other words, academic reco.1u and motivation were ranked as 
the most important factors to consider when making a decision 
to setback or to drop a student for academic reasons. 

For the nonacademic type decisions, the student's 
attitude/motivation was the most important factor for both 
nonacademic decisions. The student's military record was 
ranked very closely te student motivation for the nonacademic 
drop from training decision, and to a lesser extent for the 
nonacademic setback decision. Also, personal information was 
valued more strongly than the other student factors for the 
nonacademic type auestions. Several student factors were 
ranked very closely together and comprised a mid- ranged group 
for a given type decision. The mid-ranged group included the 
remaining student factors, with tue exceptions of those noted 
above, and the two lowest ranked student factors. The 
student's ASVAB scores and whether or not the student was a 
high school graduate (HSDS), were consistentl,’ ranked as the 


least important student factors. 


tu 
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The student's motivation was ranked the highest for 
three of the four possible decisions and it was ranked second 
for the fourtr “ccision. Even though there is agreement about 
the value of evaluating student motivation (as will be shown 
below in the discussion of Kendall's coefficient), it is a 
highly subjective judgment that is rated by a number of 
differert factors by the ARB members. There were over 25 
different -esponses cn what corstitutes attitude/motivation. 
Table 8 presents the most con...on responses, by percentage of 
the total number ot 210 responses received. Nany of the ARB 


memcers gave more than one response. 


TABLE & 


HOW MOTIVATION IS JUDGED 
(Question ¢&) 


How Go you judge motivation (besides night study). 


Participation in class 20% 
Seeking help from instructors 19% 
Completing homework 17% 
Demonstrates extra effort 15% 
Comments from instructors 10% 
Rewrites class notes 1% 
Miscellaneous 7% 


Class participation was the most frequentiy menticned 


measure of student attitude. It is foliowed by the group of 


tid 
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measures shown on the table. Other, less common responses 
included whether or not the student makes eye contact at the 
ARB, how sharp the student looks in uniform, and the "gut 
feeling" of the ARB member. 

The student's ASVAB scores were ranked very low in 
importance for any type ot decision being made. This seems 
surprising because the ASVAB scores are used as an entrance 
screen for most schools. Table 9 presents the responses for 
concerning the usefulness ASVAB scores are in determining if 
the student's academic problems are legitimate or really a 
lack of motivation. The data are based on percentages of the 
G1 people who responded to the question. Over half of the 
respondents thought a student's ASVAB scores would be useful 
in judging student motivation. However, this contradicts the 


low mean rankings given to it by the same individuals. 


TABLE 9 


VALUE OF THE STUDENT'S 
ASVAB SCORES TO THE ARB (Question 6) 


Do ASVAB scores help you determine whether a student's 
academic problems are real or due to lack of motivation? 


ASVAB scores are helpful 52.6% 
ASVAB scores are not helpful 36.8% 
Not sure 10.6% 


It would appear that, conceptually, ARB members see 


value in ASVAB scores as an aid in decision making, which is 
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reflected in Table 9. However, when compared to other sources 
of information on a relative basis, ASVAB scores are ranked at 
a very low level (see Table 7). 

The ARBs obtain information about the student from 
many sources. As stated earlier, some schools have a member 
of the ASMT's instructional staff sit on the ARBs as a voting 
member. Other schools call the ASMT whenever the student is 
having academic problems to discover if the student is also 
having other problems that may have a contributing influence 
to the student's academic proklems. The ARBs may also have 
the student's military record present at the board. The 
student's military record would contain information about the 
student's military performance, specifically noting any 
infractions the student might have committed. In some of the 
smaller schools, the academic instructors usually hear when a 
student is having problems at the ASMT. 

When a student comes before an ARB, one of the first 
objectives is to determine the kinds of personal problems the 
student is having, if any. A student's personal problems may 
include family, financial, and medical problems. Table 10 
presents the kinds of personal information that could help an 
ARB make a decision whether to setback or drop a student. 
Since ARB members could give more than one answer to this 


question, percentages are based on 245 responses. 





TABLE 10 


PERSONAL INFORMATION USED 
BY THE ARB (Question 5) 


What kind of personal information about the student might 
help you make a decision during an ARB? 


A student's personal problems 56% 


Attitude in class, 


study habits, effort 14% 
Background (family, jobs, hometown) 7% 
Education background 4% 


Misc. (substance abuse, friends, 
after hours' habits, stress, 
depression, and goals) 19% 


How a student is judged as a future sailor may have an 
affect on the ARB's decision. This factor was ranked between 
fourth and sixth as shown on Table 7. Table 11 presents the 
types of factors considered in making judgments about the 
student as a future sailor. The ARB members provided over 45 
different factors used to make this judgment. The most common 
responses are shown on Table 11. The percentages shown on the 
table are based on 250 responses. 

As described earlier, these data represent an average 
of all survey respondents. Thus, the next step in analyzing 
the data was to investigate the extent to which people agree 
on this set of averaged rankings. A Simple inspection of the 
data indicates that there is not perfect agreement. For exam- 


ple, while motivation was ranked very highly for three out of 
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TABLE 11 


FACTORS CONSIDERED WHEN MAKING 
PROFESSIONAL JUDGMENTS (Question 8) 


How do you judge whether someone will make a good sailor? 


Shows a positive attitude to complete 
the training 33% 


Motivation, willingness to work, 
desire to excel, initiative 19% 


Appearance and military bearing 17% 


Behavior, class performance, 
study habits, follows direction, 


night study, asks questions 13% 
Military record 7% 
Various personality traits 6% 
Respectful to seniors 5% 


‘the four decisions, not everyone rated it as first (the values 
shown on Table 7 range from 2.27 to 2.39). However, it is not 
clear how much disparity there is. 

The Kendall's W provides a quantification on the level 
of agreement. The Kendall's coefficients ranged from a 
moderate level of agreement for the academic setback decision 
W= .42, to a higher level of agreement for the nonacademic 
drop from training decision W = .64. In other words, the 
Kendall's Statistic applied to these data indicates that there 
is a level of agreement that is significant, or non-randon, 


for all decisions. Further, there is a higher level of 
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agreement among decision makers when dropping a student for 
nonacademic reasons as compared to other decisions. 
2. Analysis of School-Level Responses 
The remaining discussion of the level of agreement 
will focus on the four groups of schools. Particular 
attention will be paid to any differences that exist between 
the schools' rankings of the student factors. The Friedman's 
Two-Way Analysis of Variance by Ranks and Kendall's W 
Coefficient of Concordance were significant for all schools 
at the one percent level unless noted. 
a. Setback for Academic Reasons 

The schools! patterns of ranking the student 
factors (see Appendix C) were similar to the command-wide 
pattern. For all schools, the highest ranked student factors 
were the student's academic record and the student's 
attitude/motivation. The mid-ranged student factors were also 
Similarly ranked with the exception of RM-A school. RM-A 
school gave the student's ASVAB scores slightly higher mean 
and relative rankings. The other schools ranked the ASVAB 
scores as lowest in importance. Also, RM~A school valued the 
student's military record and their own professional judgment 
less than the other school groups. 

The levels of agreement within the schools were 
statistically significant indicating that there is a non- 
random basis for ranking the student factors among the 


school's ARB members. The least amount of agreement occurred 
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within the 3300-level schools (W = .43), while the AES had the 
highest level of agreement (W = .54). 
b. Setback for Nonacademic Reasons 

The data tables discussed in this section are 
presented in Appendix D. The RM-A school data were not 
statistically significant, most probably due to the small 
number of respondents, therefore the results from RM-A school 
will be omitted from the discussion. 

The schools! patterns of ranking the student 
factors were similar to the command-wide rankings, with the 
exception of the AES. The student's attitude/motivation was 
the highest ranked student factor for all the schools, and the 
student's military record was the second highest ranked 
student factor. The AES agreed with the ranking of the 
student's attitude/motivation as the highest student factor, 
but differed on the next two most important student factors. 
They valued their own professional judgment about the student 
and, to a slightly lesser extent, recommendations made to the 
board, much more than the other school groups. The remainder 
of the mid-ranged student factors were patterned similarly to 
the command-wide rankings. Also, all the schools ranked the 
student's ASVAB scores and HSDG as the least important of the 
student factors. 

For this type of decision, the levels of agreement 


were close to each other and the command wide rating with the 
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exception of the AES, which had a high level of agreement 


(W = .73). 
c. Drop for Academic Reasons 

Once again the schools' patterns of ranking the 
student factors were very similar to the command-wide rankings 
(see Appendix E). The student's academic record and the 
student's attitude/motivation were ranked closely as the most 
important student factors. The mid-ranged student factors 
were ranked similarly to the group as a whole, and ASVAB 
scores and HSDG ranked as the least important student factors 
among all the schools. 

The levels of agreement across the schools varied 
from RM-A school with the least level of agreement (W = .47) 
to the highest level at the AES (W = .59). There is slightly 
more certainty about the ranking of the student factors for 
the academic drop from training decision than there was for 
the academic setback decision. 

d. Drop for Nonacademic Reasons 

The data table summarized in this section is 
presented in Appendix F. The RM-A school data consisted of 
only two responses, therefore these results will not be 
discussed. 

The schools! patterns of ranking the student 
factors closely matched the command-wide pattern of student 
factor rankings. The student's military record and the 


student's attitude/motivation were ranked closely and were the 
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two most important student factors. The schools ranked the 
mid-ranged student factors similarly, and ASVAB scores and 
HSDG were the least important student factors. 

This type of decision produced the highest levels 
of agreement among the schools' ARB members. The 3300-level 
schools had the lowest level of agreement (W = .62), while the 
AES had the most agreement (W = .77). 

Table 12 summarizes the Kendall's W Coefficient of 
Concordance statistics for the command-wide analysis and then 
for each school group. Overall, the analyses of the data from 
each separate school show the following trends: 

1. RM-A school data were omitted from the discussions of 
nonacademic decisions due to the small number of 
respondents. RM-A instructional personnel declined to 
respond to these items because, at their ARBs, they do 
not evaluate the students for nonacademic reasons. 

2. Agreement was only slightly higher within schools as 
compared to results produced by the command-wide 
analysis, with the exception of the AES. The AES had 
consistently higher agreement across all types of 
decisions. 

3. There is more agreement among the ARB members concerning 
the importance of student factors for the decisions to 
drop a student from training as compared to setback 
decisions. 

4. There is more agreement among ARB members concerning the 


importance of student factors for nonacademic as 
compared to academic decisions. 
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TABLE 12 


AGREEMENT COEFFICIENTS BY SCHOOL GROUP 
AND TYPE OF DECISION 


Command- 3300- 

Decision Wide RM-A IC/DP-A level AES 
Academic 

Setback ~42 ~45 -46 -43 54 
Nonacademic 

Setback -50 247 -50 -48 -73 
Academic 

Drop -50 47 ~52 55 59 
Nonacademic 

Drop .64 .79 .65 .62 ony 


C. ADDITIONAL ATTRITION~-RELATED QUESTIONS 

There is a growing concern by people involved with Navy 
enlisted training that, due to the addition of a SAC for 
voluntary disenrollment, there will be an increase in the 
number of students desiring to disenroll from training. A 
realistic job preview (RLJP) is one instrument that has been 
used to prevent that situation from occurring by portraying 
the perspective workplace through lectures, books, videos, 
etc. The ARB members were asked what effect they thought a 
RLJP would have in preventing attrition at their school. 
Their responses are given in Table 13. At least 60 percent of 
the ARB members thought a RKLJP could be useful or would be 
very useful in preventing attrition at their school. The RLJP 


received the most support from the 3300-level schools and the 
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least support from the AES (obviously, a RLJP would have less 


value ina "C" School). 


TABLE 13 
VALUE OF A RLJP (Question 29) 


How useful would a realistic job preview for this rating be 
in preventing attrition for any reason? 


RM-A IC/DP-A 3300 AES 
Not useful 15.4% 17.0% 11.8% 40.0% 
Could be useful 46.2% 23.4% 11.8% 20.0% 
Very useful 38.4% 59.6% 76.4% 40.0% 
N 13 46 a7 10 


How the instructors and ARB members feel about the 
‘@ifficulty of their curriculum may affect their opinions about 
the students. Table 14 presents the percentages of ARB 
members who feel that the curriculum at their school is either 


too hard, too easy, or about right. 
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TABLE 14 


DIFFICULTLY OF THE 
SCHOOL'S CURRICULUM (Question 28) 


In consideration of what your students will have to do when 


they eventually perform in their rating, rate the curriculum 
here. 


RM-A IC/DP-A 3300 AES 
a. too hard 8.3% 31.3% 0 9.1% 
b. too easy 75.0% 8.3% 0 81.8% 
c. about right 16.7% 60.4% 100% 9.1% 
N 12 48 17 11 


The final question asked the ARB members whether their 
school's attrition had gone up, down, or stayed about the 


same, and why. The responses are presented in Table 15. 


TABLE 15 


ARB MEMBER'S PERCEPTIONS 
ABOUT THE CHANGE IN ATTRITION (Question 24) 


In the time that you have been here, has attrition gone up, 
down, or stayed about the same? Why? 


RM-A IC/DP-A 3300 AES 

Stayed the same 0% 9.8% 29.4% 100% 
Down, due to: pressure 

or lowered standards 83.3% 51.2% 0% 0% 
Down, due to: improved 

students or methods 16.7% 9.8% 64.7% 0% 
Gone up 0% 29.2% 5.9% 0% 
N 12 41 17 11 


43 


About 83 percent of the respondents from RM-A school and close 
to 51 percent of the respondents from IC/DP-A school indicated 
that attrition had gone down. The two reasons given for the 
decline in attrition were lowered grading standards and the 
pressure they had received from their superiors to lower 
attrition. Twenty nine percent of the respondents from IC/DP- 
A school stated that attrition had increased. Conversely, 
over 66 percent of the 3300-level schools respondents 
indicated that attrition had gone down due to improved 
training methods and extra effort from the instructors. All 
of the AES respondents stated that their attrition has 
remained about the same. 

It is interesting to note that the RM-A and IC-A schools 
are among the 15 schools that have high attrition, and also 
have members who are the most concerned about pressure to 
reduce attrition through reduced lowered standards. These 
instructors have apparently made some assumptions concerning 
attrition policy. 

While the data collected for the last three questions 
presented here are only indirectly related to attrition 
reporting, they are nonetheless highly relevant to the 
decisions made at ARBs. For example, an instructor who 
perceives that standards have been lowered to reduce attrition 
may be influenced in two ways. First, if the instructor 
believes that standards have already been lowered for course 


tests (or curriculum), he/she may be more likely to be 
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unnecessarily stringent in standards applied to the decision 


to setback or drop. The result could be inappropriate 
attrition. 

On the other hand, if the instructor perceives incorrectly 
that the concern with reducing attriti 1 is such that it is 
his/her job to apply lowered standards, that person may play 
a part in creating a problem ctuat doesn't exist. The result 
in this situat*on would be to reduce attrition at the expense 
of quality student output--tne very outcome of concern to a 
number of the instructors surveyed. 

Thus, indirect factors can affect attrition rates in non- 
optimal ways. These issues would seem to merit additional 


exploration. 


IV. CONCLUSIONS AND RECOMMENDATIONS 


This research examined the school's attrition process, 
specifically focusing on those areas that could result in 
either inaccurate attrition reporting or decreased 
standardization among the attrition reporting schools. The 
study identified three general areas: (1) procedural 
differences, (2) differences in the levels of agreement 
concerning various student factors, and (3) ARB memser's 


perceptions. 


A. PROCEDURAL DIFFERENCES 

The ARB procedures differ across’ schools. Those 
differences may contribute to decreased standardization in 
attrition reporting among the schools. Most of the 
differences appear t>» be due to varying interpretations of 
CNTECHTRA INST. 1540.46A, while others exist because of tke 
school's policy, i.e., the chairman as the only menber of 
assign the SACs, not assigning a nonacademic SAC because it is 
an Academic Review Board. Further standardizing the ARB 
procedures and restating the goals of the ARBs should be 


considerea. 


B. LEVELS 01 AGREEMENT 
Without an explicit policy governing what student 


information should be considered, there exist remarkable 
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similarities among the schools concerning the value certain 
student factors contribute to the ARB's decision making 
process. However, the ARB member's judgment about the 
characteristics that make-up some of those student factors 
varies. Also, the ARB members within the schools have 
moderate to high levels of agreement concerning the importance 
of the student factors. It is recommended that CNTT evaluate 
the student information that is currently being considered at 
the ARBs to determine if this is what ARBs should consider 
when making a decision to drop a student from training. 
Further, CNTT could create a rudimentary decision aid 
highlighting what student information their ARBs_ should 
consider and how much weight to assign each piece of 
information. This type of instrument could help all ARBs to 
evaluate their students based on a similar scale, and yet 
still give the ARBs the flexibility they desire to evaluate 


unique cases. 


C. ARB MEMBERS' PERCEPTIONS 

The board member's perceptions about the school's 
effectiveness in performing its mission may affect the 
accuracy of attrition reporting. Specifically, perceptions 


about course difficulty as well as perceptions of lowering 


standards to meet attrition goals may influence instructor 


morale and ARB decision making. 








Finally, the adequacy of current ARB decisions is unknown. 
It may be that the ARBs are making correct decisions. To 
determine this, two follow-on studies could be conducted. 
First, the performance of marginal students, as identified by 
test scores, could be examined in the fleet. Do they perform 
as well as the other sailors? Second, the cost-effectiveness 
of setting back students as opposed to dropping them from 
training could be examined. Specifically how many setbacks 
should be given? 

This study is the initial attempt at analyzing the 
intricacies of decision making at ARBs. Follow-on studies 
should continue this complex analysis and further the efforts 
of standardizing attrition reporting and ensuring an accurate 


NITRAS database. 








APPENDIX A 


INSTRUCTIONAL STAFF REPRESENTATION BY SCHOOL 


SCHOOL TOTAL TOTAL PERCENT STAFF 
STAFF STUDIED 
RM-A 102 14 13.7 
IC-A 62 44 71.0 
DP-A 8 4 50.0 
MS-A 33 6 18.2 
PM/ML-A 10 3 30.0 
MR-A 20 9 45.0 
AES 209 1i 563 


APPENDIX B 


ARB QUESTIONNAIRE 


The purpose of this survey is to look at how your 
organization conducts its Academic Review Boards. We are 
interested in understanding how decisions are made. In other 
words, what kind of information do you use in making a 
decision? We are particularly interested in the importance 
you give to different factors and any unique factors you may 
consider. This will allow us to better understand the 
decision making process. 


Your answers are anonymous, your command will not have any 
access to these questionnaires, and any information reported 
will be aggregated so no one answer will be singled out. 


GENERAL INFORMATION 
Rank/Paygrade 

Years of service 
Time at this command 


Approximate number of times you 
have sat on an ARB 


Approximate number of times you have 
served as chair of an ARB 


THE ARB PROCESS 


Please consider the specific types of decisions described 
below and in the spaces next to each factor indicate the 
following: 


a. How important each type of information is in 
leading to that particular decision. Use a 1-5 scale where 
not at all important; rarely used 
somewhat important 
average importance 
very important 
extremely important; critical factor 


OB WDM PR 
Wow wot wy 
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b. The rank of each of the factors compared to the others 
for that particular decision. You will have ranks 1-9 if you 
use only the factors we have suggested, or more if you can 
think of factors to add on that we have neglected to include. 


We encourage you strongly to try to add factors anywhere you 
can to make sure we have an accurate understanding of the ARB 
process. 


1. For your first set of ratings and rankings, consider a 
typical situation (we know there are unique situations, try to 
focus on the average) in which the board decides that a 
student should be setback for academic reasons. Now evaluate 
how important the following factors were in helping you reach 
this decision: 


Importance Rank 





Academic record 
Military record 


Dersonal information 
about the student 


ASVAB scores 
Amount of night study 


Recommendations made to 
the board 


Your professional judgment 
about whether this person 
will make a good sailor 
Student attitude/motivation 
High school graduate (or not) 


Other: 
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2. For the next set of ratings and rankings, consider the 
situation in which the decision is made to setback the student 
for non-academic reasons. 


Importance Rank 


Academic record 
Military record 


Personal information 
about the student 


ASVAB scores 
Amount of night study 


Recommendations made to 
the board 


Professional judgement 
about whether this person 
will make a good sailor 
Student attitude/motivation 


High school graduate (or not) 


Other: 


3. Now consider the situation in which the decision is made 
to drop a student from training for academic reasons. 


Importance Rank 





Academic record 
Military record 


Personal information 
about the student 


ASVAB Scores 
Amount of night study 


Recommendations made to 
the board 


Professional judgment 

about whether this person 
will make a good sailor 
Student attitude/motivation 
High school graduate (or not) 


Other 


4. Now consider the situation in which the board decides to 
drop a student from training for non-academic reasons. 


Importance Rank 


Academic record 


Military record 


Personal information 
about the student 


ASVAB scores 
Amount of night study 


Recommendations made 
to the board 


Professional judgment 

about whether this person 
will make a good sailor 
Student attitude/motivation 


High school graduate (or not) 


Other 
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5. What kind of personal information about the student might 
help you make a decision during an ARB? 


6. Do ASVAB scores help you determine whether a student's 
academic problems are real or due to lack of motivation? 


7. %$If you have two students with academic problems and 
everything is the same about these students except that one 
has average ASVAB scores and the other's are high, are you 


(circle your answer) 
Equally likely to attrite both 
More likely to attrite the student with average ASVABs 
More likely to attrite the student with high ASVABs 


omnomE') 


8. How do you judge whether someone will make a good sailor? 


9. How do you judge motivation (besides night study)? What 
Kinds of questions do you ask to determine motivational 
problems? 


What are some of the things that students say that would lead 
you to think that a student has a motivational (vice academic) 
problem? 


10. What questions do you ask before or during an ARB to get 
information about the student that is not reflected in the 
records? 
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11. During the interview with a student, what might convince 
you to vote to attrite instead of setback? 


12. Assume that you have only the academic record of a 
student coming to a board and that you can have only two other 
sources of information to make your decision. What would 
those two other factors be? 


13. During an ARB, about how much time is spent on each 
student? 
Give a range (max/min) and an average. 


14. About how often are there disagreements among the board 
members in arriving at a decision? Give a percentage. 


15. Briefly note the most common causes of disagreement and 
the ways in which they might be resolved. 


16. During the ARB, when one member has more influence than 
the others in shaping the decision, is this because that 
member is 


(circle your answer) 
a. More experienced in the ARB process 
b. More familiar with the student at the board 
c Most senior 
a A naturally dominant personality 


About what percent of the time is there one person who 
dominates the board processes for any of these reasons? 
percent. 
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17. After any disagreements are discussed and a decision has 
been made, indicate the percent of the time you feel 





a. Satisfied with the decision made: percent. 
b. That you still disagree with the decision: 
percent. 


18. Where did you learn how to participate in academic review 
boards? List all sources and check the one that provided you 
with the most/best information. 


STUDENT ACTION CODES 


19. Which of the SACs are confusing or hard to use in any 
way, and why? 


20. If you could add more SACs to those you have available to 
you (in order to increase the accuracy of the system), what 
would they be? 


21. Is there any reason you avoid using particular codes? 
Which ones, and why? 


22. Is there any reason you would lean toward using either an 
academic or non-academic drop code for a person who appeared 
to be about equally unable and unmotivated to complete the 
course? 


Which type code would you use and why? 
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SCHOOL INFORMATION 


23. What are the most common causes of attrition at this 
school? 


24. In the time that you have been here, has attrition gone 
up, down, or stayed about the same? 


If up, why? 


If down, why? 


25. How much communication is there here between the Military 
Training Divisions and the academic sections on student 
progress? 


(circle one) 
a. None 
b. Occasional 
c. Considerable 


26. From what you have seen, about what percentage of 
students with waivers are eventually dropped from training? 


About percent. 


27. What is the maximum number of setbacks you would give any 
student? 


Is that number based on your own feelings, 
from your command? 





or guidance 


28. In consideration of what your students will have to do 
when they eventually go out to perform in their rating, is the 
curriculum here 


(circle your answer) 


a. too hard 
b. too easy 
c. about right 


29. About how often do you think the problems leading to 
attrition are a result of a student having unrealistic/ 
inaccurate expectations of what the job/rating involves? Give 
a percentage that reflects your best guess. 


percent. 


30. At this point, please add anything that we have not 
included that will help us to have a complete understanding of 
ARBs, student action codes, and the way you do business at 
this school. 


THANK YOU FOR YOUR TIME. 
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Student 
Factor 


acadrec 
milrec 


persinfo 
asvab 
nghtstady 
rectobrd 
prfjdgmnt 
stdtmot 
hsdg 

N 

df 


Friedman's 
Chi square 


Kendall's 
coefficient 


Chi square 


APPENDIX C 


SETBACK FOR ACADEMIC REASONS 


RM-A 


1/2. 
8/6. 


3/4. 
6/5. 
5/4. 
4/4. 
7/6. 
27 3 


O77: 


39 
82 


18 


43 


92 


39 


04 


07 


75 


BY SCHOOL GROUP 


IC/DP-A 


272% 
6/5. 


4/4. 


8/5. 


3/4 


7/5. 


5/5. 


1/2 


9/7 


46 


169. 
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34 
55 


98 


96 


~92 


721 


30 


-29 


-95 


-46 


21 


-43 


62.40 


42.18 


254 


47.30 





Student 
Factor 


acadrec 
milrec 
persinfo 
asvab 
nghtstdy 
rectobrada 
prfjdgmnt 
stdtmot 
‘hsdg 

N 

ar 


Friedman's 
Chi square 


Kendall's 
coefficient 


Chi square 


APPENDIX D 


DROP FOR ACADEMIC REASONS 
BY SCHOOL GROUP 





RM-A IC/DP-A 3300 AES 
1/2.38 SW ay See 1/1.94 2/3.14 
7/6.12 7/5.51 7/6.03 7/6.23 
4/5.08 4/5.10 4/4.75 6/5.23 
8/6.35 8/6.09 8/7.00 8/7.09 
5/5.31 3/4.61 S/pEST7 A895 
3/3.81 5/5.34 6/5.14 3/3.82 
6/5.65 6/5.49 5/5.06 5/5.00 
2/2.62 2/2.48 2/2.83 1/2523 
9/7.69 9/8.28 9/8.08 9/8.32 

13 46 18 11 

& 8 8 8 
42.95 16€.37 70.49 44.87 
47 52 £55 59 
48.95 190.96 78.82 51.54 
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APPENDIX E 


SETBACK FOR NONACADEMIC REASONS 
BY SCHOOL GROUP 


Student 

Factor RM-A IC/DP-A 3300 AES 
acadrec 5/4.75 6/4.96 7/5.35 6/6.00 
milrec Oat fe eee 2/3.46 PIBETS 4/4.05 
persinfo 4/4.12 3/3.68 3/4.09 5/4.25 
asvab 8/7.12 8/7.33 8/7.38 8/7.85 
nghtstdy 7/6.87 7/6.27 6/4.97  7/6.55 
rectobra 1/2.62 4/4.16 5/472 S73:.540 
prfjdgmnt 6/5.62 5/484 EILEDY 2/3.05 
stdtmot 3/3.50 1/2.44 1/3435 1/1.80 
hsdg 9/7.25 9/7.87 9/8.15 9/8.05 
i 4 45 17 10 
df 8 & 8 8 
Friedman's 

Chi square 13.72* 160.02 56.28 53.13 
Kendall's 

coefficient -47 -50 -48 -73 
Chi square 15.20* 179.13 65.33 58.65 


* Nonsignificant p > .05 





APPENDIX F 


DROP FOR NONACADEMIC REASONS 


BY SCHOOL GROUP 


Student 

Factor RM-A 
acadrec 5/5.25 
milrec 4/4.75 
persinfo 1/2.25 
asvab 8/7.75 
nohtstdy 7/7.00 
rectobra 2/2.75 
prfjdgmnt 5/5.25 
stdtmot 3/3.25 
hsdg 9/7.75 
N 2 

af 8 
Friedman's 

Chi square 9.73% 
Kendall's 

coefficient .79 
Chi square 12.69% 


* Nonsignificant p > .05 


IC/DP-A 
6/5.15 
1/2320 
573579 
8/7.26 
7/6.38 
5/4.91 
4/4.66 
2/2.29 
9/6 .35 
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214.75 


-65 


237.66 
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75.92 


-62 


84.50 


~77 


61.46 
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